Interactive Virtual Relighting and Remodeling of Real Scenes

In our method, real scene geometry is first approximately reconstructed from photographs. Additional images are taken from a single viewpoint with a real light in different positions to estimate reflectance. A filtering process is used to compensate for inaccuracies, and per image reflectances are averaged to generate an approximate diffuse reflectance image for the given viewpoint, removing shadows in the process. An example of extracted reflectance is given in Fig. 1(b). Other approaches could be used [2] for reflectance extraction. This reflectance estimate is used to initialise a global illumination hierarchical radiosity system, representing real-world secondary illumination. Direct illumination from lights is calculated separately using ray-casting and a table for efficient reuse of data where appropriate. Fig. 1(c) shows shadow reprojection using the estimated reflectance (b).